Cloudserver adaptation post arsenal refacto #6044

benzekrimaha · 2026-01-14T17:31:35Z

bert-e · 2026-01-14T17:31:39Z

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-01-14T17:31:47Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

Issue: CLDSRV-826

codecov · 2026-01-14T17:39:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.25%. Comparing base (247b008) to head (87202c4).
⚠️ Report is 109 commits behind head on development/9.3.
✅ All tests successful. No failed tests found.

Additional details and impacted files

see 2 files with indirect coverage changes

@@                 Coverage Diff                 @@
##           development/9.3    #6044      +/-   ##
===================================================
- Coverage            84.29%   84.25%   -0.04%     
===================================================
  Files                  204      204              
  Lines                12962    12962              
===================================================
- Hits                 10926    10921       -5     
- Misses                2036     2041       +5

Flag	Coverage Δ
file-ft-tests	`67.39% <ø> (-0.04%)`	⬇️
kmip-ft-tests	`28.08% <ø> (ø)`
mongo-v0-ft-tests	`68.62% <ø> (-0.13%)`	⬇️
mongo-v1-ft-tests	`68.63% <ø> (-0.11%)`	⬇️
multiple-backend	`35.23% <ø> (ø)`
sur-tests	`35.56% <ø> (-0.81%)`	⬇️
sur-tests-inflights	`37.40% <ø> (-0.04%)`	⬇️
unit	`69.89% <ø> (ø)`
utapi-v2-tests	`34.23% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

SylvainSenechal · 2026-01-15T10:26:09Z

tests/functional/raw-node/test/GCP/bucket/get.js

+    return gcpClient.send(command)
+        .then(data => callback(null, data))
+        .catch(err => {
+            if (err && err.$metadata && err.$metadata.httpStatusCode &&


no need to check if err exists here. I think err.$metadata.httpstatus code will always be defined.
Probably same for statusCode, gotta call the api once to verify but I feel like we shouldn't have to check if its defined or not

SylvainSenechal · 2026-01-15T10:36:19Z

tests/functional/raw-node/test/GCP/object/completeMpu.js

    before(done => {
        config = getRealAwsConfig(credentialOne);
        gcpClient = new GCP(config);
+        gcpClient.listObjects = (params, callback) => {


Just to understand : this was available in Arsenal GcpService but you just removed it and now you are redefining it in all these files, why not keeping it in Arsenal ? I think the GcpService already has several functions that are used for tests only 🤔
Or is it because you want to reassign the statusCode ?

Upon looking : consider updating the gcpClientRetry function : check err.$metadata.statuscode == 429 on top of the existing check err.statusCode == 429.
This way, can keep listObjects in Arsenal. (unless there are other reason I didn't see)

even if we added the $metadata.httpStatusCode check inside gcpClientRetry, we still need bespoke error munging in these raw-node tests. Rather than keeping a half-working helper in Arsenal plus a bunch of local overrides, I’d rather keep the shared GcpService minimal and let the tests own whatever extra plumbing they require.

in tests, we should not need a generic wrapper to remap errors : each test is a single scenario, we can update that test to check the excepted result (both error or success)

even if we really absolutely need a helper, just introduce a regular function but do not modify the gcpClient object

can't we migrate each test instead of keeping this yet another AWS sdk v2 lookalike api?

SylvainSenechal

Don't forget to use real arsenal version in package json
Just one comment to address about the gcpClient.listObjects, maybe can consider a utility function at least, or even better keep it in Arsenal

some more comments to address

As these functions are only used in GCP tests, we decided to remove them from gcpService. This commit updates the tests accordingly using the new send command directly and enabling us to drop the wrapper functions. Issue: CLDSRV-826

francoisferrand

in addition to the comments overall, there is something weird in these test: we use GCP client from arsenal, yet at the same time there is a re-implementation of GCP without the sdk (c.f. makeGcpRequest, retryGcpRequest...)

this should probably be changed, to do things consistently: either with GCP client (to maximize code reuse), ~~or with a custom implementation,~~ or directly with the GCP sdk (if we want to ensure the test code is independent form the production code, and failure would thus only happen due to change in production code)

francoisferrand · 2026-01-20T07:13:39Z

tests/functional/raw-node/test/GCP/bucket/get.js

+gcpClient.listObjects = (params, callback) => {
+    const command = new ListObjectsCommand(params);
+    return gcpClient.send(command)
+        .then(data => callback(null, data))
+        .catch(err => {
+            if ( err.statusCode === undefined) {
+                // eslint-disable-next-line no-param-reassign
+                err.statusCode = err.$metadata.httpStatusCode;
+            }
+            return callback(err);
+        });
+};
+
+gcpClient.getBucket = (params, callback) =>
+    gcpClient.headBucket(params, (err, res) => {
+        if (err) {
+            if (err.statusCode === undefined) {
+                // eslint-disable-next-line no-param-reassign
+                err.statusCode = err.$metadata.httpStatusCode;
+            }
+            if (err.$metadata && err.$metadata.httpStatusCode === 404) {
+                // eslint-disable-next-line no-param-reassign
+                err.name = 'NoSuchBucket';
+            }
+            return callback(err);
+        }
+        return callback(null, res);
+    });
+


why keep these "legacy" functions, and not migrate the actual tests to the new SDK api?
since this PR is focused, it does not seem like a large change...

francoisferrand · 2026-01-20T07:14:57Z

tests/functional/raw-node/test/GCP/object/completeMpu.js

        config = getRealAwsConfig(credentialOne);
        gcpClient = new GCP(config);
+        gcpClient.listObjects = (params, callback) => {
+            const command = new ListObjectsCommand(params);


same as above, should migrate each test? (and avoid duplicating this piece of code)

…ands lint fixup

…ands fixup NoSuchBucket default returned error name

francoisferrand · 2026-01-28T16:17:32Z

tests/functional/raw-node/test/GCP/bucket/get.js

-                process.stdout.write(`err in creating bucket ${err}\n`);
-            }
-            return done(err);
+        const command = new CreateBucketCommand({


there used to be a gcpRequestRetry() : no problem to stop using it, but don't we need to setup retry somehow? or is it useless and handled in the client now?

retries are now handled by the sdk itself

francoisferrand · 2026-01-28T16:19:11Z

tests/functional/raw-node/test/GCP/bucket/get.js

    before(done => {
-        gcpRequestRetry({
-            method: 'PUT',
-            bucket: bucketName,
-            authCredentials: config.credentials,
-        }, 0, err => {
-            if (err) {
-                process.stdout.write(`err in creating bucket ${err}\n`);
-            }
-            return done(err);
+        const command = new CreateBucketCommand({
+            Bucket: bucketName,
        });
+        gcpClient.send(command)
+            .then(() => done())
+            .catch(err => {
+                process.stdout.write(`err in creating bucket ${err}\n`);
+                return done(err);
+            });


before(async () => { const command = new CreateBucketCommand({Collapse commentComment on line Bucket: bucketName, }); await gcpClient.send(command); });

francoisferrand · 2026-01-28T16:21:08Z

tests/functional/raw-node/test/GCP/bucket/get.js

    after(done => {
-        gcpRequestRetry({
-            method: 'DELETE',
-            bucket: bucketName,
-            authCredentials: config.credentials,
-        }, 0, err => {
-            if (err) {
-                process.stdout.write(`err in deleting bucket ${err}\n`);
-            }
-            return done(err);
+        const command = new DeleteBucketCommand({
+            Bucket: bucketName,
        });
+        gcpClient.send(command)
+            .then(() => done())
+            .catch(err => {
+                process.stdout.write(`err in deleting bucket ${err}\n`);
+                return done(err);
+            });
    });


after(async () => { const command = new DeleteBucketCommand({ Bucket: bucketName, }); await gcpClient.send(command) });

francoisferrand · 2026-01-28T16:22:26Z

tests/functional/raw-node/test/GCP/bucket/getVersioning.js

    const config = getRealAwsConfig(credentialOne);
    const gcpClient = new GCP(config);

    beforeEach(function beforeFn(done) {


make function async and use await when sending command (same for afterEach)

francoisferrand · 2026-01-28T16:25:30Z

tests/functional/raw-node/test/GCP/bucket/getVersioning.js

    it('should verify bucket versioning is enabled', function testFn(done) {
        return async.waterfall([
-            next => makeGcpRequest({
-                method: 'PUT',
-                bucket: this.test.bucketName,
-                authCredentials: config.credentials,
-                queryObj: { versioning: '' },
-                requestBody: xmlEnable,
-            }, err => {
-                if (err) {
-                    process.stdout.write(`err in setting versioning ${err.code}`);
-                }
-                return next(err);
-            }),
+            // Enable versioning using the official SDK client
            next => {
-                gcpClient.getBucketVersioning({
+                const command = new PutBucketVersioningCommand({
                    Bucket: this.test.bucketName,
-                }, (err, res) => {
-                    assert.equal(err, null,
-                        `Expected success, but got err ${err}`);
-                    assert.deepStrictEqual(res.Status, verEnabledObj);
-                    return next();
+                    VersioningConfiguration: { Status: 'Enabled' },
                });
+                return gcpClient.send(command)
+                    .then(() => next())
+                    .catch(err => next(err));
+            },
+            // Verify using GetBucketVersioningCommand
+            next => {
+                const command = new GetBucketVersioningCommand({
+                    Bucket: this.test.bucketName,
+                });
+                return gcpClient.send(command)
+                    .then(res => {
+                        assert.deepStrictEqual(res.Status, verEnabledObj);
+                        return next();
+                    })
+                    .catch(err => next(err));
            },
        ], err => done(err));
    });


almost every line is changed, best use async and shorten the test:

it('should verify bucket versioning is enabled', async () => { await gcpClient.send(new PutBucketVersioningCommand({ Bucket: this.test.bucketName, VersioningConfiguration: { Status: 'Enabled' }, })); const res = gcpClient.send(new GetBucketVersioningCommand({ Bucket: this.test.bucketName, })); assert.deepStrictEqual(res.Status, verEnabledObj); });

francoisferrand · 2026-01-28T16:34:49Z

tests/functional/raw-node/test/GCP/object/deleteMpu.js

+                },
+            err => {
+                if (err && (err.name === 'SlowDown'
+                    || err.$metadata?.httpStatusCode === 429)


bad indent, this is not at the same indent level as 'err' (nested expression)

Suggested change

|| err.$metadata?.httpStatusCode === 429)

|| err.$metadata?.httpStatusCode === 429)

or better yet just keep it on previous line, the line does not seem that long?

francoisferrand · 2026-01-28T16:42:37Z

tests/functional/raw-node/test/GCP/object/getTagging.js

+        function createBucket(attempt) {
+            const cmd = new CreateBucketCommand({ Bucket: bucketName });
+            gcpClient.send(cmd)
+                .then(() => {
+                    bucketCreated = true;
+                    return done();
+                })
+                .catch(err => {
+                    if ((err.name === 'SlowDown'
+                        || err.$metadata?.httpStatusCode === 429)
+                        && attempt < maxAttempts - 1) {
+                        const delay = Math.pow(2, attempt) * 1000;
+                        process.stdout.write(
+                            `SlowDown creating bucket, retrying in ${delay}ms (attempt ${attempt + 1})\n`);
+                        return setTimeout(() => createBucket(attempt + 1), delay);
+                    }
+                    process.stdout.write(`err in creating bucket ${err}`);
+                    return done(err);
+                });
+        }
+
+        createBucket(0);


can't this be written much more cleanly with async.retry or even just modifying the gcpRequestRetry to take both a command and client and repeatedly send the command if needed... (that function would not really be specific to gcp, but provide a good separation between retry logic and the test)

francoisferrand · 2026-01-28T16:48:19Z

tests/functional/raw-node/utils/gcpUtils.js

-
 const genUniqID = () => uuidv4().replace(/-/g, '');

-function gcpRequestRetry(params, retry, callback) {


should still keep the retry logic isolated in some helper, instead of re-implementing directly in the tests

helper may need to support both async & callbacks (depending on where it is used)

async function gcpRetry(gcpClient, cmd, retry, cb) { if (cb) { return callbackify(() => gcpRetry(gcpClient, cmd, retry))(cb); } [...] return await gcpCient.send(cmd) }

francoisferrand · 2026-01-28T16:48:42Z

tests/functional/raw-node/utils/gcpUtils.js

-        return callback(null, res);
-    });
-}
-


if we really need helper (listObjects, ...) they should be added here

francoisferrand · 2026-01-28T16:49:17Z

package.json

    "@hapi/joi": "^17.1.1",
    "@smithy/node-http-handler": "^3.0.0",
-    "arsenal": "git+https://github.com/scality/Arsenal#8.3.0-preview.1",
+    "arsenal": "git+https://github.com/scality/Arsenal#aed443f02efebeb3f5c2e74786baadb5c19eaded",


should be updated with arsenal 8.3.0 now that it is released

bert-e · 2026-01-29T07:58:00Z

Branches have diverged

This pull request's source branch improvement/CLDSRV-826 has diverged from
development/9.3 by more than 50 commits.

To avoid any integration risks, please re-synchronize them using one of the
following solutions:

Merge origin/development/9.3 into improvement/CLDSRV-826
Rebase improvement/CLDSRV-826 onto origin/development/9.3

Note: If you choose to rebase, you may have to ask me to rebuild
integration branches using the reset command.

lint fixup

callback handling

benzekrimaha force-pushed the improvement/CLDSRV-826 branch from 1ac5fba to 5baf746 Compare January 14, 2026 17:32

bump latest up to date arsenal

a3a207a

Issue: CLDSRV-826

benzekrimaha force-pushed the improvement/CLDSRV-826 branch from 5baf746 to ea9cb65 Compare January 14, 2026 17:34

benzekrimaha marked this pull request as ready for review January 14, 2026 17:34

benzekrimaha requested review from a team, SylvainSenechal, delthas, francoisferrand and maeldonn and removed request for delthas and maeldonn January 14, 2026 17:35

SylvainSenechal reviewed Jan 15, 2026

View reviewed changes

SylvainSenechal previously approved these changes Jan 15, 2026

View reviewed changes

benzekrimaha changed the title ~~Improvement/cldsrv 826~~ Cloudserver adaptation post arsenal refacto Jan 19, 2026

benzekrimaha force-pushed the improvement/CLDSRV-826 branch from b7e25d5 to bd529bb Compare January 19, 2026 16:38

francoisferrand reviewed Jan 20, 2026

View reviewed changes

update arsenal tag

b5f9d21

benzekrimaha force-pushed the improvement/CLDSRV-826 branch from 10955a6 to 26dab1f Compare January 23, 2026 16:58

benzekrimaha added 2 commits January 27, 2026 16:48

step1: use send with gcp client on bucket tests instead of old helpers

e3f5042

step2: use send with gcp client on object tests instead of old helpers

9c34aa1

benzekrimaha force-pushed the improvement/CLDSRV-826 branch from c3be2d7 to 9c34aa1 Compare January 27, 2026 15:49

benzekrimaha added 3 commits January 27, 2026 17:49

step3: remove legacy list and get and replace with sdk v3 commands

8fd93f6

fixup! step3: remove legacy list and get and replace with sdk v3 comm…

37c70c1

…ands lint fixup

fixup! step3: remove legacy list and get and replace with sdk v3 comm…

b476392

…ands fixup NoSuchBucket default returned error name

francoisferrand reviewed Jan 28, 2026

View reviewed changes

benzekrimaha added 3 commits January 29, 2026 12:34

bucket test async functions introduction

2955d37

fixup! bucket test async functions introduction

bf2edf8

lint fixup

fixup! bucket test async functions introduction

87202c4

callback handling

	\|\| err.$metadata?.httpStatusCode === 429)
	\|\| err.$metadata?.httpStatusCode === 429)


		const genUniqID = () => uuidv4().replace(/-/g, '');

		function gcpRequestRetry(params, retry, callback) {

Cloudserver adaptation post arsenal refacto #6044

Are you sure you want to change the base?

Cloudserver adaptation post arsenal refacto #6044

Uh oh!

Conversation

benzekrimaha commented Jan 14, 2026

Uh oh!

bert-e commented Jan 14, 2026

Hello benzekrimaha,

Uh oh!

bert-e commented Jan 14, 2026

Waiting for approval

Uh oh!

codecov bot commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

SylvainSenechal Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SylvainSenechal Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SylvainSenechal left a comment

Choose a reason for hiding this comment

Uh oh!

francoisferrand left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bert-e commented Jan 29, 2026

Branches have diverged

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Jan 14, 2026 •

edited

Loading

SylvainSenechal Jan 15, 2026 •

edited

Loading

SylvainSenechal Jan 15, 2026 •

edited

Loading

francoisferrand left a comment •

edited

Loading